The present invention relates generally to a distributed storage system. More particularly, the present invention relates to a geographically distributed storage system configured to have a hierarchical peer to peer (P2P) architecture.
In the era of Web Computing, highly scalable storage systems are needed. Such a storage system needs to meet the demands for storing and accessing information, wherein such demands increase exponentially because web services typically start from a small number of users and expand rapidly when the services are known by other users around the world. Due to this characteristic of web computing, such a storage system needs to be scalable from multiple perspectives including performance, capacity, manageability, and so on.
Geographically distributed storage systems are required in order to reduce usages of network bandwidth, reduce response time to access data, increase availability of services, and so on. In a case where storage systems are located at multiple geographical locations, it becomes much more difficult to design, install, manage and maintain the system compared with a storage system located in one location. Traditional storage systems require a large number of highly skilled engineers working at the site of the storage systems. However, it is too expensive to employee so many engineers and successfully continue the business because customers do not pay a lot for just storing and accessing data. Thus, it is necessary for storage services to be provided with less labor costs.
Peer to peer (P2P) file sharing technology can be used to reduce such labor costs dramatically. Typical P2P software is installed in each personal computer at home and used for sharing files among tens of thousands of computers around the world. Such computers are called, nodes. Files are replicated and stored in multiple nodes. However, even if some number of nodes have been turned off or have failed, users can still continue to use the file sharing service on the nodes that are still on or that have not failed. There is no central control server or dedicated administrator or highly skilled engineer to maintain the service.
Traditional P2P technology is not concerned with the physical locations of the nodes or network connectivity among the nodes. Because of this, network bandwidth is not used efficiently. For example, when one user living in Japan has requested to read a file, the file might have been stored in and be transferred from a node on the other side of the world such as Brazil via an expensive submerged cable. Networking traffic caused by P2P has been consuming a large part of network bandwidth for some years and this has been a big problem for internet service providers (ISPs). Some ISPs decided to restrict the use of P2P software or charge more for such use. It is an important problem for ISPs to reduce networking traffic caused by P2P software.
When geographically distributed storage systems adopt P2P technology as a basic architecture for managing the storage systems, the same problem as to P2P file sharing will occur. It is demanded by online storage service providers to store files in the location where the files are accessed frequently so the files are accessed within a local network so as not to cause unnecessary network traffic across the Internet.
There are some prior arts disclose the use of P2P technology with data location control. The following three US published patent applications are examples of such techniques.
US Patent Application Publication No. 2004/0085329A1 discloses a system by which each node measures latencies of communications from the landmark nodes, the system groups the nodes which have similar latencies to the landmark nodes and the system stores files among the nodes in the specific group. It is assumed that the nodes in a group can be considered to be located in the same location if the measured latencies are close to each other. However this assumption is not always correct because latency may vary depending on network conditions. Alternative methods to group nodes which are located in same location is required.
US Patent Application Publication Nos. 2004/0249970A1 and 2004/0054807A1 disclose a system by which an administrator specifies a location of each node and a hierarchical organizational topology of the locations in manual and the system groups the nodes within same location, create a hierarchical topology among the groups and stores data among the nodes in the specific location. It is assumed that the hierarchical topology among the nodes has been already known by the administrator. However when the number of the nodes in the system becomes very large and the number of the locations has also increased, it is difficult to know the updated topology information around the Internet. Alternative method to create a hierarchical topology among the groups is required.